up: [[Statistical Bias]] Tags:: # Multiple comparisons fallacy If you do a hypothesis test on data that has a chance of having a false positive than your probability of getting a Type 1 error goes up because there is now a much higher chance of getting significance while still claiming the probability the whole analysis will be correct 95% of the time. ![[Pasted image 20221005151338.png]] #### What is the family wise error rate? The family wise error rate (FWER) measures the rate at which the probability of a type 1 error changes based on the increase in the number of independent decisions. It's equation is below: ![[Pasted image 20221005151437.png]] #### How do you correct for it? Use the Bonferroni correction which involves taking the testwise alpha (your alpha without correction) and dividing by the number of independent decisions being performed. ^74d9f9 Then you take this new alpha and plug it into the family wise equation. Here is an example with a .05 testwise alpha level and k = 3. ![[Pasted image 20221019145547.png]] This correction becomes less and less effective the larger the number of independent decisions because the probability of a Type 2 error becomes massive. #### How does the Bonferroni correction effect statistical power? It increases the chance of a Type 2 error because the alpha gets smaller for each test making the [[Statistical Power]] smaller. Related: ___ # Resources